Method for producing a synthesized stereoscopic image

ABSTRACT

Methods, devices and systems are provided which produce a synthesized stereoscopic image of a source image by forming at least two images from a source image where at least one image has been modified relative to the source image such that the images have a different spacial appearance than each other. A wide variety of modifications may be made to the source object in order to form modified stereo images. These modifications may involve modifying one or more images relative to the source image. For example, at least one image may be magnified, reduced, or rotated in the X, Y and/or Z plane relative to the source image. Alternatively or in addition, the position of one or more elements of one of the images may also be changed relative to the source image. Alternatively or in addition, at least one of the images may be transformed relative to the source image using a function which alters the position of elements of the image along the X or Y axis relative to the source image. In one variation, the function is a distorting algorithm, for example an elliptical or aspheric distorting algorithm which is not dependent on the depth information of the source image.

RELATIONSHIP TO COPENDING APPLICATIONRELATED APPLICATIONS

This application is a continuation-in-part of “METHOD AND DEVICE FORSTEREOSYNTHESIS,” claims the benefit of priority of U.S. ProvisionalApplication Ser. No.: 60/024,900; filed: Aug. 30, 1996 which isincorporated herein by reference.

FIELD OF THE INVENTION

The present invention relates to methods, devices and display systemsfor converting source images to synthesized stereoscopic images and morespecifically for forming two or more images from a source image where atleast one image of the two or more images has been modified relative tothe source image such that synthesized stereoscopic images are formed.

BACKGROUND OF THE INVENTION

Stereoscopic imagery for television and computer monitors is performedby presenting a different image to each eye of the observer. Images foreach eye may be created and presented electronically, electroopticallyand/or purely optically such that the human observer perceives a threedimensional image. Using these methods, it is possible to create complexthree dimensional presentations which may be used in a wide variety ofapplications.

Most systems for producing stereoscopic imagery have depended on the useof true stereo pairs of images created by complex and costly optical orcomputer systems. Although there have been attempts to convert twodimensional images to three dimensional images using field delay withimage shifting (e.g. U.S. Pat. No. 5,510,832), such conversions have notproduced three dimensional imagery having sufficient quality. Forexample, it is not possible to create a quality three dimensional imageof a still image using prior art methods since field delay may berequired. Other transformations of such pairs of images from oneencoding method to another has been also been difficult and costlybecause they generally require depth information and computation.Furthermore, it has been necessary to generate stereo pairs of imagesusing two separate cameras or a single camera with special lenses. Sucharrangements are costly and difficult to use.

A need therefore exists for a method and device for providingstereoscopic images which can be done rapidly and inexpensively. Thedevice and method should also be able to form the stereoscopic imagewithout substantially degrading the image or color.

SUMMARY OF THE INVENTION

The present invention relates to methods, logic, data signals, recordeddata, devices and systems for use in providing a synthesizedstereoscopic image from a source image by forming two or more modifiedstereo images from the source image where at least one of the modifiedstereo images has been modified relative to the source image such thatat least two of the modified stereo images have a different spacialappearance than each other. The stereoscopic image formed may be used ina holographic display.

As used herein, “different spacial appearance” refers to a difference inthe size and/or orientation of two images and/or a difference in thespacial relationship between elements of one image and the correspondingelements of the other image. The different spacial appearance can beperceived when the two images are displayed and compared. However, theimages with a different spacial appearance can be expressed as dataencoding the images which, when displayed, have a different spacialappearance. The images can also be expressed as data encoding a sourceimage in combination with one or more algorithms for modifying thesource image and producing two images which have a different spacialappearance.

As also used herein, “modified stereo images” refers to the two or moreimages derived from the source image where at least one image of the twoor more images has been modified relative to the source image such thatat least two of the two or more images have a different spacialappearance. These modifications to the images do not required knowledgeof the relative depts of particular elements of the image. Modifiedstereo images can be in the form of two or more images being displayedas well as data encoding the two or more images which when displayedwould have a different spacial appearance. Modified stereo images can bein the form of data encoding a source image in combination with one ormore algorithms for modifying the source image and producing two or moreimages which have a different spacial appearance.

A wide variety of modifications may be made to the source object inorder to form the modified stereo images. These modifications mayinvolve modifying one or more of the images forming the modified stereoimages relative to the source image. For example, at least one image ofthe modified stereo images may be magnified, reduced, or rotated in theX, Y and/or Z plane relative to the source image. Alternatively or inaddition, the position of one or more elements of one of the modifiedstereo images may also be changed relative to the source image.Alternatively or in addition, at least one image of the modified stereoimages may be transformed relative to the source image using a functionwhich alters the position of elements of the image along the X or Y axisrelative to the source image. In one variation, the function is adistorting algorithm, for example an elliptical or aspheric distortingalgorithm.

As used herein, the source image may be any two or three dimensionalimage, holographic image or set of two or three dimensional images.Examples of source images include, but are not limited to motion picturefilm, photographs, computer images, video and tomographic data sets,such as those derived from MRI or CT data. The source images may berecorded, for example on VHS video tape, betacam SP or D1 tape,nonlinear edit system, time base corrector, computer floppy disc,computer hard drive, RAM, CDROM, laserdisc and DVD. The source image mayalso be provided via a live broadcast, video signals, or generated by asoftware program. The source image can also be an analog or digital setof data corresponding to one or more two or three dimensional images.

In one embodiment of the invention, a method is provided for producing asynthesized stereoscopic image by displaying modified stereo imageshaving a different spacial appearance, and viewing the modified stereoimages through stereo viewing glasses which transmit the spatiallydifferent images to left and right eyes of the viewer to form asynthesized stereoscopic image of the source image, such as a VRheadset. In one variation of this embodiment, an image display is usedto transmit the modified stereo images to the stereo viewing glasses. Ina further variation, image signals encoding the modified stereo imagesare transmitted to the image display by a device which converts recordeddata encoding the modified stereo images into image signals which arereceived by the image display. In yet another variation, image signalsencoding the modified stereo images are transmitted to the image displayby an image processor which converts a signal encoding the source imageinto signals encoding the modified stereo images. According to thisvariation, conversion of the signal encoding the source image into thesignals encoding the modified stereo images is performed in real time.Conversion may alternatively be delayed. Conversion of the image signalsmay be an analog to analog, analog to digital, digital to digital,digital to analog, and/or in combination with optical to optical signalconversion.

Further, according to this variation, the signal encoding the sourceimage may include a signal which provides instructions to the imageprocessor regarding how to convert the signal encoding the source imageinto signals encoding the modified stereo images. These instructions maybe simultaneously broadcast and may include, for example, image elementby image element instructions or image frame by image frameinstructions.

In another embodiment of the invention, a method is provided forproducing a synthesized stereoscopic image using stereo viewing glasseswhich form modified stereo images. In this embodiment, the methodincludes displaying on an image display a source image, and viewing thesource image through stereo viewing glasses, the stereo viewing glasseshaving left and right lenses, at least one of the lenses modifying thesource image to produce at least two images having a different spacialappearance (modified stereo images) when viewed through the lenses. Inone variation of this embodiment, modifying the source image to producemodified stereo images having a different spacial appearance isperformed in real time. Alternatively, the modified stereo images arecreated and stored prior to being displayed. Modifying the source imagecan involve an analog to analog, analog to digital, digital to digital,digital to analog, and/or in combination with optical to optical signalconversion.

In another embodiment of the invention, a method is provided forconverting a signal encoding a source image into recorded signalsencoding two or more spatially different images which form a synthesizedstereoscopic image of the source image when displayed in combination. Inthis embodiment, the method includes taking a signal encoding a sourceimage, forming two or more image signals encoding the source image,modifying at least one of the two or more image signals such that atleast two of the image signals encode images which have a differentspacial appearance than each other, and recording signals encoding themodified stereo images. The signals encoding the modified stereo imagesmay be recorded on a variety of different media including, for examplemagnetic media, VHS video tape, Betacam SP or D1 tape, nonlinear editsystem, frame store, computer floppy disc, computer hard drive, RAM,CDROM, laserdisc, DVD, MPEG and other optical or digital device.Modifying at least one of the image signals can involve an analog toanalog, analog to digital, digital to digital, digital to analog, and/orin combination with optical to optical signal conversion.

In another embodiment of the invention, a method is provided forsynthesizing a stereoscopic image from a source image. According to thisembodiment, the method includes taking a signal encoding a source image,forming two or more image signals encoding the source image, modifyingat least one of the two or more image signals such that the two or moreimage signals encode at least a pair of images having a differentspacial appearance, displaying on an image display the spatiallydifferent images encoded by the modified image signals, and viewing theimage display through stereo viewing glasses which transmit thespatially different images to a left and right eye of the viewer to forma synthesized stereoscopic image of the source image. In one variationof this embodiment, modification of the image signals is performed inreal time. Modifying at least one of the image signals can involveanalog to analog, analog to digital, digital to digital, digital toanalog, and/or in combination with optical to optical signal conversion.

In a variation of this embodiment, the two or more image signalsencoding the source image includes a signal which provides instructionsto an image processor regarding how to modify at least one of the two ormore image signals. These instructions may include, for example, imageelement by image element instructions or image frame by image frameinstructions. These instructions can be simultaneously broadcasted theimage signals and can be combined with the image signals. For example,the instructions can be embedded into the signal, for example, in thevertical sync.

The present invention also relates to signals encoding two or moreimages which have a different spacial appearance which are derived fromthe same source image and which, when viewed in combination, form asynthesized stereoscopic image of the source image. These signals may beanalog or digital signals. In one variation, the two or more imagesinclude two images which have a different spacial appearance than eachother. In another variation, the two or more images include three imageswhich have a different spacial appearance than each other. These signalsmay be transmitted signals or signals which are recorded on a recordingmedia. In one particular embodiment, these signals are derived from acomputer generated image which has been modified according to thepresent invention.

It is envisioned that larger bandwidth systems, such as HDTV, will beable to transmit and receive separate signals for each modified imageused to form the synthesized stereoscopic image. Accordingly, thepresent invention is also intended to be used in combination with suchhigher bandwidth systems where two or more signals each encoding amodified stereo image is transmitted and/or received.

The present invention also relates to a recording of syntheticstereoscopic images according to the present invention. The recordingincludes a recording medium having recorded thereon two or more signalsencoding at least two images having a different spacial appearancewhich, when viewed in combination, form a synthesized stereoscopicimage. The recording may encode an analog or digital signal. Anyrecording media capable of storing dual image signals may be used,including, but not limited to VHS video tape, betacam SP or D1 tape,nonlinear edit system, computer floppy disc, computer hard drive, RAM,CDROM, laserdisc and DVD.

The present invention also relates to an image processor for convertinga signal encoding a source image into a signal encoding a synthesizedstereoscopic image. In one embodiment, the image processor includes asignal entry port for receiving a signal encoding a source image, logicfor converting the source image into two or more image signals encodingthe source image where at least one of the two or more image signals ismodified such that the image signals encode two or more images having adifferent spacial appearance, and a signal exit port for transmittingthe modified image signals.

The logic may act to convert the source image to the synthesizedstereoscopic image in real or non-real time. The logic may beprogrammable or controllable by the user to form different modifiedstereo images. For example, the user can program the logic to makecertain elements of an image or certain images appear closer or fartheraway. Alternatively or in addition, the logic may be designed todynamically control how the image processor transforms the source imagebased on the source image. This dynamic control may be on a program byprogram, image by image or element by element basis. For example, thelogic may include certain image element recognition protocols whichcause certain elements to be selectively brought into the foreground orbackground when detected. In one embodiment, these protocols can be usedto sense whether an image corresponds to a close-up or wide-angleperspective so that the image maybe modified accordingly. The logic mayalso include certain movement recognition protocols which detectsmovement of certain elements and causes the image to be modified in aparticular manner in response to the detected movement. This adaptationis particularly useful in pan and scan applications where it isdesirable to keep certain image elements, such as an actor, centered inthe image.

In a variation of this embodiment, the image processor also receives oneor more signals which accompany the signal encoding the source image andprovides instructions to the image processor regarding how to modify thesource image. These instructions may include, for example, image elementby image element instructions or image frame by image frameinstructions. For example, the instructions may direct image 1 to begiven a first type of modification, image 2 to be given a second type ofmodification, image 3 to be given a third type of modification, etc.Each type of modification may be a global modification for the image orprovide for multiple different modifications for different elements inthe image.

The present invention also relates to stereo viewing glasses forconverting a source image into a synthesized stereoscopic image. In oneembodiment, the stereo viewing glasses include a first and second lensfor receiving the source image, and an optic associated with at leastone of the first and second lenses which modifies the spacial appearanceof the source image such that the pair of lenses provide a pair ofimages having a different spacial appearance which, when viewed incombination, form a synthesized stereoscopic image. The stereo viewingglasses can include only optics, the images being formed by a separateimage display. Alternatively, the stereo viewing glasses can include oneor more image displays. These stereo viewing glasses can alsoincorporate existing optical techniques which have been used to createstereo effects.

In conjunction with forming synthesized stereoscopic images, the variousembodiments of the present invention can be employed in a variety ofapplications where it is desirable to create a three dimensionaldepiction. For example, the present invention can be used to transformimages formed in existing computer programs into stereoscopic images.Accordingly, the present invention can be used to enhance the appearanceof cascaded windows being stacked over each other. Different windows canalso be made to appear closer or farther from the observer.

The various embodiments of the present invention can also be employed incombinations with known pan and scan techniques to enhance theeffectiveness of three dimensional imaging. For example, motion anddepth can be introduced into a still picture by panning across the stillpicture. Meanwhile, depth can be introduced by modifying the pannedimage according the present invention. In combination, the modified,panned stereoscopic image has a significantly greater three dimensionalappearance.

Another application of the present invention is in the area of medicaltomography such as MRI and CT scans where a series of images are takenat different known depths. By distorting each image according to itsknown depth, the series of two dimensional images can be converted intoa series of three dimensional images which more accurately reflect thetomography of the imaged body location. In addition to creating a threedimensional image based on an overlay of a set of two dimensionalimages, a sense of depth can be exaggerated in each two dimensionalimage, thus making it easier to distinguish elements in each image.

The present invention can also be used in medical endoscopy where twodimensional images taken from an endoscope are converted into threedimensional images by distorting the two dimensional image provided bythe endoscope. By providing a stereoscopic endoscopic image as opposedto a two dimensional endoscopic image, the doctor is better able todetermine the relative position of different objects in the field ofview of the endoscope.

The present invention can also be used in combination with quicktimevirtual reality applications where three dimension imaging is introducedinto or enhanced in such applications.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A-I illustrate a series stereo pairs where at least the rightimage has been modified in a different manner than the left image.

FIG. 1A illustrates a reduced right image.

FIG. 1B illustrates a magnified right image.

FIG. 1C illustrates a rotated right image.

FIG. 1D illustrates a right image which is skewed along the X axis as afunction of the Y axis.

FIG. 1E illustrates a right image which is parabolically skewed alongthe X axis as a function of the Y axis.

FIG. 1F illustrates a right image which has a spherical distortion incausing the center of the image to appear at a different position in theZ axis relative to the X-Y plane.

FIG. 1G illustrates a portion of the right image being magnified.

FIG. 1H illustrates a center portion of the right image being magnified.

FIG. 1I illustrates a portion of the left image being magnified and acenter portion of the right image being reduced.

FIG. 2 illustrates a typical stereoscopic display system.

FIG. 3A is a timing diagram showing the transmission of left and rightimages to the eyes of a user of a time sequential stereoscopic system.

FIG. 3B illustrates an over-under format for displaying modified stereoimages.

FIG. 3C illustrates a time sequence of a series of over-under imageframes as in FIG. 3B where either a first or second image is displayedin each image frame.

FIG. 4 shows the effect of optical lenses used in one embodiment of theinvention.

FIG. 5 illustrates a generalization of a system for producingsynthesized stereoscopic images.

FIGS. 6A-6G represent particular embodiments of the system illustratedwith regard to FIG. 5.

FIG. 7A illustrates a set of tomographic images.

FIG. 7B illustrates a set of stereo pairs of the tomographic imagesillustrated in FIG. 7A.

FIG. 7C illustrates a set of stereo pairs of the tomographic imagesillustrated in FIG. 7A where particular images in the set of tomographicimages are modified relative to each other to provide an improvedstereoscopic description of those particular images in the set.

FIG. 8A illustrates a set of bowling pins.

FIGS. 8B-8E illustrate a series of two-dimensional tomographic images ofthe set of bowling pins where each image shows a different row of pins.

FIG. 8F illustrates a stereoscopic image of the bowling pins createdfrom the tomographic images illustrated in FIGS. 8B-8E.

FIG. 8G illustrates a top down view of the image illustrated in FIG. 8F.

FIG. 8H illustrates the set of bowling pins illustrated in FIG. 8A and asecond viewing axis.

FIGS. 8I-8L illustrates a series of two-dimensional tomographic imagesof the set of bowling pins taken along the second viewing axis.

FIG. 8M illustrates a stereoscopic image of the bowling pins createdfrom the tomographic images illustrated in FIGS. 8I-8L.

FIG. 8N illustrates a top down view of the image illustrated in FIG. 8M.

FIG. 9 illustrates different images on a computer monitor where eachimage has a user selectable three dimensional appearance.

FIG. 10 illustrates examples of different pairs of wire meshes which canbe generated by modifying the wire frame mesh illustrated in the figureas the initial wire mesh.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

The present invention relates to the discovery that the mind perceivestwo or more spatially different versions of the same image presented tothe left and right eye as a synthesized stereoscopic image, i.e., as athree dimensional image. The present invention exploits this realizationthrough methods, logic, data signals, recorded data, devices andstereoscopic imaging systems which convert a source image into asynthesized stereoscopic image by forming two or more images from thesource image and modifying at least one of those images relative to thesource image such that at least two of the images do not have the samespacial appearance as each other with or without regard to theirdistance.

In previous stereoscopic systems, synthesized stereoscopic images havebeen created using image shifting in combination with time delay, i.e.,by controlling the time at which a same image is provided to the leftand right eye of a viewer. An example of a time delay stereoscopicsystem is described in U.S. Pat. No. 5,510,832 which is incorporatedherein by reference. By contrast to time delay based stereoscopicsystems, the present invention creates synthesized stereoscopic imagesby forming two or more images which have a different spacial appearance.The use of images with a different spacial appearance according to thepresent invention can be done in combination with time delay orindependent of time delay or motion.

Stereoscopic images have also been formed previously with the use ofcomputers which calculate how the left and right eyes of a viewer wouldperceive each element of an image given the relative position of theleft and right eyes to each element. The amount of computation requiredby this approach significantly limits its practicality for real timeconversion of source images to stereoscopic images. In addition, it isnecessary to know in advance the true depth of each element in the imageat a given time in order to do the computation. By contrast,stereoscopic images are formed according to the present invention bymodifying the source image without regard to simulating how each elementof the source image would appear to the left and right eyes of the userif those elements were actually being seen by the user. Instead,elements of the source object or the entire source object are modifiedby an algorithm which causes the element or entire image to appear at acertain distance in the resultant stereoscopic image. By avoiding theneed to precisely calculate how each element of the source object wouldappear to the left and right eyes of the user, the computational demandsinvolved in the present invention are significantly reduced as comparedto computer generated stereoscopic images. The reduced computationaldemands involved in the present invention enable stereoscopic images tobe generated according to the present invention in real time. Ininstances where the relative depth of elements in an image are known,the present invention can modify these images to simulate the knowndepth without having to precisely calculate how each pixel of the imageshould appear in order to simulate depth.

Two or more images with different spacial appearances can be formed bymagnifying, reducing, stretching (X axis or Y axis) rotating (X-Y plane)and/or tilting (rotating in X-Z and/or Y-Z planes) one or more of theimages. A different spacial appearance can also be formed by modifyingthe relative position of elements in all or a portion of one or more ofthe images.

In one embodiment, modified stereo images are formed by taking a sourceimage, forming two or more images from the source image, andtransforming at least one of the two or more images using a function(f(x,y)) which alters the position along the X and/or Y axis of all orsome of the objects appearing in the source image. The function may alsoserve to alter the image on a line by line basis when differentfunctions can be used in different lines. Examples of transformingfunctions include distorting algorithms such as elliptical and asphericalgorithms, enlargements or offsets. In a preferred embodiment, thefunction is a non-linear distortion along the X and/or Y axes.

Any combination of the above modifications may be made on all or anyportion of the right and/or left eye image in all or only some of thefields. More than one modification can also be made to the same image.In the case of stereoscopic imaging involving multiple frames (e.g.,video), all or some of the multiple frames may have modified stereopairs.

In embodiments where the present invention is used to generate threedimensional stereo graphics of a computer image, the modified stereoimages may be formed by taking a source image and mapping the sourceimage onto a wire mesh and distorting the image according to a function,such as those described herein and illustrated with regard to FIGS.1A-I.

FIGS. 1A-I illustrate a series of modified stereo image pairs where atleast the right eye image has been modified relative to the sourceimage, illustrated as a uniform horizontally and vertically orientedgrid. Although pairs of images are illustrated, it should be noted thatsets of three or more images can also be formed.

FIG. 1A illustrates a reduced right eye image. FIG. 1B illustrates amagnified right eye image. FIG. 1C illustrates a rotated right eyeimage. FIG. 1D illustrates a right eye image which is skewed along the Xaxis as a function of the Y axis. FIG. 1E illustrates a right eye imagewhich is parabolically skewed along the X axis as a function of the Yaxis. FIG. 1F illustrates a right eye image which has a sphericaldistortion in causing the center of the image to appear at a differentposition in the Z axis relative to the X-Y plane. FIG. 1G illustrates aright eye image where a portion of the image has been magnified. FIG. 1Hillustrates a right eye image where the center of the image has beenmagnified. FIG. 1I Illustrates a left eye image with a magnified centerportion and a right eye image with a reduced center portion. FIGS. 1A-Iare intended to be illustrative of the various modifications which canbe made to the source image in order to form modified stereo image pairsand are not intended to be exhaustive. These and other modifications toa source image to produce modified stereo image pairs can be created bya variety of digital or optical devices in real or non-real time withoutdeparting from the scope of the present invention.

In one particular embodiment, a pair of modified images are tilted orrotated in the X and/or Y and/or Z plane 10 percent or less relative tothe other image. In another particular embodiment, a pair of modifiedimages are magnified or reduced 10 percent or less relative to eachother. In yet another particular embodiment, one of a pair of modifiedimages is delayed one field or less relative to the other eye image. Inall of the above embodiments, a convincing depth synthesis of thesemodified stereo image pairs occurs.

One example of a preferred transformation is the simultaneous warps ofFIGS. 1B, 1F and 1H with 1 field or frame delay, horizontal offset of 3%and multiple target tracking with motion detection determining thedirection of delay (i.e., on field 1 or 2), a 3% magnification of oneimage with the image mapped on a sphere twice the diameter of the screenwith its axis displaced 2/3 up the height of the screen and with theseeffects oscillating from right to left eye image at 10 Hz.

Careful adjustment of all parameters of the video signal (orphotographic, printed or film or computer image) as well as minimizingof parallax is necessary for good depth, minimal ghosting andcomfortable viewing.

With ordinary stereo imaging, it is necessary to present the right eyeimage only to the right eye and the left eye image only to the left eye.If this is not done, confusing pseudoscopic images which cause eyestrainresult. A unique feature of the present invention is that it is possiblein some cases to present either image to either eye without pseudoscopy.This is made possible by the fact that the differences in the two ormore images are created by modifying one or more of the images relativeto a source image as opposed to relying on the relative spatialpositions of the images at the time of photography or videography inorder to cause the sensation of depth. Consequently, though FIG. 3A forexample shows all the transforms being performed on one side, channel orelement, in fact both sides, channels or elements may perform the sameor different transforms to different degrees and such transforms ordegrees may change over time in order to obtain maximum effect withminimum discomfort for the viewer.

A typical stereoscopic display system is illustrated in FIG. 2. Thesystem includes an image source 12, an image processor 14, an imagedisplay 16, and stereo viewing glasses 18. According to the presentinvention, the stereoscopic display system produces modified stereoimages by taking a source image and forming a two or more images whereat least one image has been modified relative to the source image andrelative to the other images of the modified stereo images.

The formation of modified stereo images can be performed by severaldifferent components of a stereoscopic display system. For example, theimage source 12 can provide image data corresponding to modified stereoimages. Alternatively, the image processor 14 can receive image datafrom the image source 12 and convert the image data into modified stereoimages according to the present invention. In this variation, the imagedata can also include instructions regarding how to modify the sourceimage, for example, element by element or image by image. Alternatively,optics may be positioned between the image display and the user whichconvert images formed by the image display 16 into modified stereoimages. For example, the lenses of stereo viewing glasses 18 may eachreceive the same image from the image display 16 and convert the imageinto modified stereo images according to the present invention. Thestereo viewing glasses may optionally include the image display 16.

It is intended for the present invention to encompass image sourceswhich provide image data encoding for modified stereo images accordingto the present invention as well as instrumentation for producing imagedata encoding for modified stereo images. The present invention alsoencompasses image processors which convert data for source images intoimage signals for modified stereo images according to the presentinvention. The present invention also encompasses optics which may beused to optically convert images produced by an image display intomodified stereo images according to the present invention.

The present invention is also intended to encompass stereoscopic imagingsystems which include an image source, image processor and/or opticswhich produce modified stereo images according to the present invention.In this regard, components of the system which are not being used tomodify a source image to produce modified stereo image pairs may bestandard equipment which is currently used in existing stereoscopicimaging systems.

The present invention is also intended to encompass methods employed bythe above components of a stereoscopic imaging system for producingmodified stereo images according to the present invention.

A significant advantage of the methods, devices and stereoscopic imagingsystems of the present invention is their ability to convert existingtwo dimensional and three dimensional motion picture films, photos,computer images, videos and single cameras with ordinary lenses into avariety of stereo formats in real time or nonreal time. In the case ofreal time conversions, the methods and devices of the present inventionmay be used to convert unrecorded image signals, for example from a livebroadcasts, video signals received via antenna or cable, or images froma computer program into a stereoscopic rendering of the program in fullcolor and in real time.

A further advantage of the present invention is the ability to convertthe source images into a series of different stereoscopic images basedon the way in which the stereo images are modified relative to eachother. Hence, the methods, devices and systems of the present inventionare designed to be programmable to provide a series of different visualeffects. In addition to the general solidizing or roundness of objects,one can make objects protrude from the screen and when combined withtarget acquisition and tracking performed on the same machinesimultaneously with the other image manipulations described here, thisstereosynthesis can be very sophisticated, real time and automatic. Anobject may be made to appear to recede into the background as it movesor becomes smaller. Further, artificial movement of an image, i.e.movement that occurs in the modified images but not in the source image,can be created by selecting a series of modification which introduce thesense of motion. Some elements in an image can be made to appear closerwhile other elements appear farther away. One can also detect foregroundobjects based on size or speed of motion and then magnify, delay,spherically warp or otherwise manipulate them selectively in real timeautomatically or for best effects, nonrealtime with more sophisticatedalgorithms or with human intervention using known means such asdigitizing tablets and the like. It is also possible by these automaticmeans or with user intervention to select people or other objects bycombining image acquisition with speech detection to create varioustypes of games or contests from ordinary or specially encoded broadcastor prerecorded video.

A further advantage of the present invention is the limited computationthat is required to convert the source images to stereoscopic images.Stereoscopic images have previously been formed by calculating how theleft and right eyes of a viewer would perceive each element of an imagegiven the relative position of the left and right eyes to each element.The amount of computation required by this approach significantly limitsits practicality for real time conversion of the source images tostereoscopic images. By contrast, stereoscopic images are formedaccording to the present invention without regard to simulating how eachelement of the source image would appear to the left and right eyes ofthe user if those elements were actually being seen by the user. As aresult, the computational demands involved in the present invention aresignificantly reduced as compared to computer generated stereoscopicimages and thus can be generated in real time.

A further advantage of the present invention is the ability of themethods, devices and systems to convert the source images into threedimensional images in a single step. In this regard, aside from thecomponent or components which are being used to form the pair ofmodified stereo images, the methods and devices of the present inventionmay be used with standard equipment. For example, except where modifiedstereo images are converted by the image viewer, full color stereoscopicimages are viewable with inexpensive dual filter glasses (amaglyphglasses), LCD shutter glasses or other devices known in the art forcontrolled image transmission to the left and right eyes of theobserver. Also, ordinary analog or digital video and/or computerhardware and/or software can be used.

A further advantage of the present invention is the color quality of thestereoscopic images produced. Because the conversion of source images tothree dimensional images according to the present invention does notinvolve modifying the color of the source images, high quality fullcolor stereoscopic images can be produced.

1. Image Source

In general, the image source 12 may be any recorded media, an imagesignal transmitter or data set which provides image data to the imageprocessor 14. Examples of sources of recorded source images which can bereadily converted into three dimensional images by the present inventioninclude two dimensional and three dimensional motion picture films,photographs, computer images, and videos. In one particular example, theimages are images generated by computer software. These image sourcesmay be in a digital or analog format or in a real image format. Examplesof recorded formats include, but are not limited to, videotape, magneticmedia such as floppy discs, digital recorded media such as digitalmemory, digital tape, compact discs, DVD and the like. Examples of imagetransmitters include broadcast antenna, cable, fiber and satellite.

According to the present invention, the image source may be a standardimage source which provides image data encoding standard two dimensionalsource images which are later processed by a component of a systemaccording to the present invention to form modified stereo image pairs.

When the image source is an unmodified image as opposed to a modifiedstereo image, the image source may include a signal which providesinstructions to an image processor regarding how to convert the signalencoding the source image into signals encoding the modified stereoimages. These instructions may include, for example, image element byimage element instructions or image frame by image frame instructions.

Alternatively, the image source may include image data creating modifiedstereo images. In this embodiment, the image source provides image datato the image processor corresponding to modified stereo images. Theimage processor receives the image data for the modified stereo images.This embodiment is a non-real time embodiment of the invention in thesense that the conversion of source images to modified stereo images isnot being performed in real time by the system.

This embodiment of the invention enables standard video recording media(video tape, laser disc, DVD) to be used to store data encoding thestereoscopic images used in the present invention. By storing the dataas opposed to converting source images to modified stereo images in realtime, the processing speed requirements of the stereoscopic imagingsystem is greatly reduced.

In embodiments where the image source provides image data encodingmodified stereo images, the image data is preferably provided in anover/under or left/right format where two of the modified stereo imagesare placed on the same frame. In this format, a first image 21 from aset of modified stereo images is taken and condensed into an upperportion (or left portion) of an image frame. Meanwhile, a second image23 from the set of modified stereo images is condensed into a lowerportion (or right portion) of the same image frame, as illustrated inFIG. 3B. The first and second images 21, 23, although each being placedin only a portion of the image frame, are preferably non-interlacedimages containing all the image data for the image. By contrast, the useof interlaced field images causes half the amount of image informationto be stored per image frame.

FIG. 3C illustrates a time sequence of a series of over-under imageframes as in FIG. 3B where either a first or second image 21, 23 isdisplayed in each image frame. Either the first or second image or theover-under image can be displayed from each image frame at a given time.Accordingly, you can have any combination of first (1) and second (2)images displayed in time sequence, (e.g., 12121212 or 11221122 or 111222or 112112 or 111112, etc.). This approach provides the system of thepresent invention with significantly greater flexibility for producingvisual effects based on displaying different combinations of first andsecond images. For example, this approach can be used to create certainvisual effects, such as the movement of an image element laterallyacross the screen. By controlling whether an image element is shown tothe left or right eye, the appearance of movement by the image elementlaterally across the image can be created or enhanced.

The use of an over/under or left/right format has further advantage ofenabling different visual effects to be performed. For example, lineaveraging between two left or two right eye frames can be performed.

In one embodiment, computer software is used as described in “UNIVERSALSTEREOSCOPIC INTERFACE,” U.S. Provisional Application Ser. No.:60/013,738; filed: Mar. 20, 1996; and “UNIVERSAL STEREOSCOPICINTERFACE,” U.S. application Ser. No.: 08/280,570; filed: Mar. 19, 1997,each of which are incorporated herein by reference. This computersoftware enables one to distinguish between and selectively displaydifferent images of a pair of images presented in an over/under format.This software is useful for increasing the range of display formatswhich can be used with the modified stereo image pairs used in thepresent invention.

2. Image Processor and Image Display

In general, the image processor 14 receives image data via a signalentry port 15. The image processor 14 includes logic for converting theimage data into image signals encoding the source image and formodifying at least one of the image signals such that the image signalsencode images having a different spacial appearance. The image processoralso includes a signal exit port 17 for transmitting the modified firstand second image signals to an image display 16 which displays realimages corresponding to the modified image signals. Examples of imagedisplays include, but are not limited to, screens associated withprojection systems, televisions, CRT screens, VR or holography.

Standard image displays such as television sets have a refresh rate of15.25 KHz which is above the frequency at which the human eye canperceive individual images. As a result, the image displays appear topresent a continuous image. Presenting alternating left and right eyeimages effectively causes an image display to have an effective refreshrate that is half the normal refresh rate of that display. If therefresh rate falls below the frequency at which the human eye canperceive images, the mind perceives the display as flickering which cancause eye strain and headaches. In order to eliminate this effect, it ispreferred to increase the rate at which lines or entire images arepresented. This can be done by line doubling, frequency doubling orframe multiplication techniques which can be performed before or afterthe source image is modified.

In the embodiment described in Section 1, the image processor receivesimage data from an image source corresponding to modified stereo image.Alternatively, the image processor can receive image data from an imagesource corresponding to a source image and produce an image signal forthe source image.

In a particular embodiment of the invention, the image processorconverts image data from an image source encoding a standard sourceimage into image signals encoding modified stereo images. In thisembodiment, the image processor takes image data from the image sourceand forms signals for images where at least one of the images has beenmodified relative to the source image such that the modified images donot have the same spacial appearance. Processing of the image data fromthe image source into modified stereo images may be done in real time ornot in real time by the image processor and recorded. The signalsinputted into the image processor and outputted by the image processormay be analog or digital signals.

In a preferred embodiment, the image processor includes logic forperforming multiple different transformations of a source image toproduce different modified stereo images based on a source image. Thedifferent modified stereo images are produced by differenttransformations of the source image to create different visual effects.For example, certain transformations increase or decrease the depth offield or field of view provided.

The different transformation performed by the image processor may beselectable by the observer, thereby providing the observer with controlover the visual effects produced. For example, the user can selecttransformations which cause certain elements of an image to appearcloser or farther away relative to their appearance in a source image.The user can also select among different transformations which introducedifferent degrees of spatially transformations to increase or decreasethe disparity perceived between close and distant elements in the sameimage. In this regard, the observer can design the appearance of theresulting stereoscopic images. The ability of an observer to customizethe stereoscopic imagery produced is an important feature of the presentinvention since how stereoscopic images are perceived varies from personto person.

The image processor can include logic which controls how the imageprocessor transforms the source image based on the source image. Thislogic can correspond to software or a computer chip encoding the logic.The logic can be used to enable the image processor to dynamicallyadjust itself to optimize the stereoscopic appearance of differentimages or elements in images. For example, the logic can be used todetect motion in the source image and modify the source image to makethe images appear closer or farther away based on the motion detected.The logic can also be used for special effects to create the appearanceof motion of elements of the image which are stationary in the sourceimage.

The logic of the image processor can also be used as an editing tool tomake certain objects appear closer or farther away. The ability tomagnify or reduce the appearance of elements within an image is animportant application of the present invention in conjunction with filmediting, such as pan and scan. This type of editing tool is also usefulin conjunction allowing an observer to customize the appearance of astereoscopic image, for example by allowing an observer to select whichimages or elements of an image to magnify, reduce or distort. Anotherapplication of this editing tool is in the computer software arena wherean observer selects which elements of a stationary computer generatedimage the observer wishes to bring into the foreground or thebackground.

For optimal depth with minimal eyestrain, it is desirable to carefullyadjust the effects on each shot of a program and to have them varytimewise. It is also desirable to design the image processor to permitthe user to control which image transformation is performed. Forinstance, the entire right eye frame or parts of a frame may be delayedone field, magnified 5 percent and horizontally shifted 7 percent whilethe right half of the image is tilted back in the Z axis 6 percent, anobject in the middle convexly projected on a spherical surface ofdiameter 3 times the screen height, while the left eye image may beuntransformed or have an object in its center magnified 8 percent, theleft half given vertical rugose transforms, the top half concavelyprojected on a dodecahedron with smallest 4 times the screen width etc.These transforms could change in kind and vary in degree and from oneeye's field to the other either slowly or as rapidly as every field withall these parameters preprogrammed or under user control.

Conversion of image data encoding a source image to an image signalencoding modified stereo images can be performed formed in real time byanalog and/or digital devices or by computer software. Conversion couldalso be done with a single video stream from digital or analog source orfrom dual streams. In cases where field delay or field advancement isdesired, this could be introduced by running two copies of the sametape, disc or digital source one or more fields out of sync or bypicking fields as desired from an analog or digital store in real ornonreal time. Common examples of such sources and stores are betacam SPor D1 tapes, nonlinear edit systems, frame store, computer hard drivesand RAM, CDROM, laserdisc and DVD.

With advanced hardware and/or software and especially with non-real timesystems, with human operator input or with automatic image segmentationand pattern recognition techniques known in the art, parts of the sourceimage can be designated for particular transforms and followed fromframe to frame by the program. For instance, a human figure can beidentified, given a five percent convex transform and one field delay inthe left eye until it disappeared from the scene or the scene changed.Parts of the image can be transformed in this manner and composited andrecomposited into the image.

Delays or advancement of one or more field or parts of fields can bepart of the synthesized difference between the images presented to thetwo eyes. Ideally, the delay will be one field or less and will dependon the direction and speed of motion. With most sequences of movingimages, such delay introduces spurious vertical parallax andpseudoscopic imagery which looks odd and creates eyestrain. Non-realtime shot to shot editing is essential for use of delay and is desirablefor all the other image transforms as well. For many sequences, no delaywill give the best results. Delay can be applied to only part of thefield instead of or in addition to the whole field.

3. Stereo Viewing Glasses

The stereo viewing glasses 18 operate in combination with the imagedisplay 16 to coordinate what images are provided to the left and righteyes 22, 24 of the observer 20. By controlling which images are providedto the left and right eyes and/or the timing of when those images areprovided, an impression of a three dimensional image can be created froma pair of two dimensional images. For example, FIG. 3A illustrates animage display 16 for use with flicker glasses to control what images areobserved by the left and right eyes of the observer. As illustrated,when a left image 28 is presented, the left lens 29 of the stereoviewing glasses 18 transmits the left image while the right lens 31blocks or absorbs the left image 28. This allows the left eye of theobserver to see the left image 28, while the right eye does not see theleft image 28. When the right image 30 is presented, the lenstransmissions are reversed so that the right eye sees the right imageand the left eye does not see the left image. By alternating between theleft and right image, the observer's brain receives both images andsynthesizes the two images into a single image which the braininterprets as having depth.

The stereo viewing glasses 18 may have optical, electrooptic ormechanical lenses. In one preferred embodiment, each lens is activatedelectronically to block the light reaching the eyes alternately. Theglasses may be gelatin, glass, dichroic polarized, anaglyphic, orholographic filters or optical or electrooptic and need only haveappropriate optical characteristics.

Although the invention is described primarily with regard to filter orshutter glasses, any other display modality may be used such asdisplaying right and left stereo pairs in frame, field, line or pixelsequential form or as simultaneous pairs for stereoscopic orautostereoscopic viewing. In one embodiment of the present invention,the images are presented autostereoscopically by known means without theuse of stereo viewing glasses. For example, the stereo viewing glassescan have polarized lens. Then, by controlling the polarization of thelight, different images can be directed to the left and right eyes.Alternatively, the stereo viewing glasses can have red/green and bluefilters on the left and right eyes respectively. Then, by transmittingone image in red and green light and the other image in blue light,different images can be selectively transmitted to the left and righteyes of the observer. Other mechanisms for presenting different imagesto the left and right eyes of the observer that are known in the art orlater developed and are intended to be used in the present invention.

In another embodiment of the invention, modified image pairs may beformed by optics positioned between the image display and the observerwhich selectively modify left or right eye images produced by the imagedisplay. For example, the image display or the stereo viewing glassesmay include optics which modify a source object produced by the imagedisplay such that modified pairs of left and right eye images areproduced. FIG. 4 shows the effect of one type of lens used in accordancewith the present invention. In this embodiment, the left eye lens 48does not transform the image produced by the image display. Meanwhile,the right eye lens 50 modifies the right eye image according to thepresent invention. The brain combines the left and right images toproduce a single image which is perceived as having depth. In thissimplest case, there is no electronic processing of the two dimensionalimage and the glasses are composed of optical elements which distort ortransform the image differently for each eye such that the viewerobtains a sensation of depth.

4. Systems for Forming Synthesized Stereoscopic Images

The following example describes different systems for formingsynthesized stereoscopic images. FIG. 5 illustrates a generalization ofthe overall system. FIGS. 6A-6G represent particular embodiments of thesystem illustrated with regard to FIG. 5.

As illustrated in FIG. 5, A represents a two dimensional data set of asource image. The data set can be an analog or digital data set. Thedata set is copied by device B which can be an optical image duplicatoror signal distribution amplifier. Signals C and C′ represent at leasttwo copies of the data set encoding the source image which are conveyedto separate sets of image processors, represented as processors D, E andF and D′, E′ and F′. A greater or lesser number of image processors canbe used than are illustrated. The data sets encoding the source imageare spatially altered by the processors according to one or morefunctions for modifying the image, such as the functions that have beendescribed herein. The signals from the processors can be resent throughthe different processors as many times as desired, as illustrated bylong arrows 81.

Device K controls the qualities of the spatial distortions introducedinto the image signals M, M′ by the processors, further modifications ofthe image signals M, M′ by devices G and G′, and/or the signals N, N′that are sent to image display devices H, H′. Devices G and G′ mayperform a variety of functions including, for example, frequencydoubling or other modifications which provide image enhancement. DeviceJ represents the different techniques for combining two or more imagesto produce stereoscopic images including, for example, sequential andoptical field multiplexing, such as shutter glasses or polarized lenses.Device J can also represent a device for recording the image signals asdigital or analog data sets.

FIG. 6A illustrates an embodiment of the system illustrated in FIG. 5where A represents any video signal source including for example, acamera, VCR, broadcast signal or DVD. Device B represents a signaldistribution amplifier. Image processing in this system is performed bya dual channel video effects device (DVE) D, D′ which is controlled bycontroller K. In this embodiment, image display devices H, H′ arevirtually reality (VR) glasses with dual video displays, one for eacheye.

FIG. 6B illustrates an alternate embodiment of the system illustrated inFIG. 5 where A represents any video signal source and device Brepresents a signal distribution amplifier. Image processing in thissystem is performed by a single DVE D which is controlled by controllerK. An optional second DVE D′ is also illustrated which can optionallyalter image signal N′ relative to the source image. In this embodiment,image display devices H, H′ are virtual reality (VR) glasses with dualvideo displays, one for each eye.

FIG. 6C illustrates another alternate embodiment of the systemillustrated in FIG. 5 where A represents any video signal source anddevice B represents a signal distribution amplifier. Image processing inthis system is performed by DVE D, D′ which is controlled by controllerK. In this embodiment, a multiplexer G field sequentially multiplexsignals M and M′. The multiplexed image signals M and M′ are displayedon video display H (e.g., a TV or computer monitor). Shutter glasses Jcombine the multiplexed images displayed on video display H. J canalternatively be polarized lenses where the video display H displaysmultiplexed images formed of light with different polarizations.

FIG. 6D illustrates another alternate embodiment of the systemillustrated in FIG. 5 where A represents any video signal source, deviceB represents a signal distribution amplifier, D, D′ represent an DVEimage processor and K represents a controller. In this embodiment, amultiplexer G1 field sequentially multiplexes signals M and M′ whileline doubler G2 increases the refresh rate at which images are displayedon video display H. Line doubler G2 can also include the functionalityof a picture enhancer for providing edge improvement. Shutter glasses Jcombine the multiplexed images displayed on video display H.

FIG. 6E illustrates a variation of FIG. 6D where the system furtherincludes a motion detector P which detects motion of elements of animage, e.g., whether a camera is panning from right to left, left toright, up, down, etc. Device P can alternately be used to detect whethera particular image element is close or distant in order to determinewhat transformation to use. Based on the type of motion detected bymotion detector P, the controller K determines which image (encoded by Mand M′) to display first. When the multiplexer G1 is used to combine theimages encoded by M and M′ in an over-under format, motion detector Pcan also by used to determine which image (encoded by M and M′) thedisplay H should display first.

The motion detector P can also be used to detect motion in an image, thedetection of which is used by the controller K to direct the processorD, D′ to modify the image signals to show that an element of an image ismoving in a particular direction. For example, the detection of anelement moving toward the screen can be used by the controller to directthe processor to modify the images to make the element appear closer inthe stereoscopic image, thereby enhancing the stereoscopic effect.

FIG. 6F illustrates an alternate variation of FIG. 6D where the systemfurther includes a motion detector P which is only connected to themultiplexer. In this embodiment, the motion detector P is used to detectmotion of elements in an image, e.g., whether a camera is panning fromright to left, left to right, up, down, etc, and uses this informationto determine which image (encoded by M and M′) the image display Hshould display first. When the multiplexer G1 is used to combine theimages encoded by M and M′ in an over-under format, motion detector Pcan also be used to determine which image (encoded by M and M′) thedisplay H should display first.

6. Stereographic Imaging of Sets of Tomographic Images

The following example illustrates the modification of a set oftomographic images to provide improved stereographic imaging of the setof tomographic images. In this example, a set of tomographic images,illustrated in FIG. 7A is taken and converted into stereo pairs of thetomographic images, illustrated in FIG. 7B, where the pair of imagesformed of each slice has a different spacial appearance relative to eachother. As illustrated in FIG. 7C, only particular images of the set oftomographic images are modified relative to each other to provide animproved stereoscopic depiction of those particular images in the set.

7. Tomographic Imaging Application

The following example describes a tomography application for the devicesand methods of the present invention for generating a synthesizedstereoscopic image from a two dimensional source image. FIG. 8Aillustrates a three dimensional object, in this case a set of bowlingpins viewed along axis 71. FIGS. 8B-8E illustrate a series oftwo-dimensional tomographic images of the set of bowling pins takenalong axis 71 where each image shows a different row of pins, intendedto represent a different depth layer of the three dimensional image. Bytaking the four tomographic images illustrated in FIGS. 8B-8E,distorting each image in relation to its relative position within thethree dimensional image, and overlaying the four images, a stereoscopicimage of the bowling pins can be created, as illustrated in FIG. 8F. Adata matrix can then be formed of the different elements (bowling pins)as they appear in the overlaid stereoscopic image. FIG. 8G illustrates atop down view of the data matrix illustrated in FIG. 8F.

Using the data matrix illustrated in FIGS. 8F and 8G, it is possible togenerate different sets of two dimensional images taken along differentviewing axes. For example, FIG. 8H illustrates the set of bowling pinsillustrated in FIG. 8A and a viewing axis 73. FIGS. 8I-8L illustrates aseries of two-dimensional tomographic images of the set of bowling pinstaken along axis 73. Using the set of two dimensional images, it is thenpossible to distort each image as a function of its position along axis73 and overlay the images to form a new stereoscopic image illustratingthe appearance of the object along the viewing axis 73, as illustratedin FIG. 8M. FIG. 8N illustrates a top down depiction of the stereoscopicimage illustrated in FIG. 8M.

As illustrated by FIGS. 8A-8N, it is possible to convert a series oftomographic images of an object and use the methods and devices of thepresent invention to create different synthesized stereoscopic images ofthe object along different viewing axes. This application of the presentinvention is particularly useful in medical applications like MRI and CTimaging where it is desirable to be able to create different threedimensional renderings of an object in order to identify the relativepositions of different elements of the object. For example, by viewing athree dimensional rendering of an object at different angles, it ispossible to more accurately determine the relative positions ofdifferent elements within an object.

In one variation of this example, an observer attaches a mechanism tohis or her head which can detect head motion. The head movementmechanism is coupled to the system for generating the stereoscopicimages and is used to direct different stereoscopic images to begenerated based on the angle or position of the observer's head, asdetermined by the head movement mechanism. This enables an observer, forexample, to turn his or her head sideways or up or down in order to viewthe stereoscopic image of the object at different angles. The system, ina preferred embodiment, is able to track the head movement and modifythe stereoscopic image accordingly in real time.

8. Stereographic Display of Computer Images

The following example describes a method for forming stereographicdisplay of a computer image. Illustrated in FIG. 9 are regions D1through D10 which represent images displayed on a computer monitoraccording to the present invention which appear to have separate threedimensional appearances when viewed using shutter glasses, polarizedplates or anaglyph techniques or other three dimensional imagingtechniques.

According to this example, the software program receive data encoding asource image as input, forms modified stereo images of the source image,and then texture maps the image by applying a function to the sourceimage, and then texture maps the image to modify the appearance of theentire source image based on that function, as opposed to forming astereographic image by calculating how each pixel of each image wouldappear to the left and right eyes of the user.

The software program can be used to independently texture map thedifferent images represented by D1 through D10 in order to createindependent stereographic appearances for each image based on theparticular modified stereo images formed, i.e., different modificationscause the images to have a different stereographic appearance. Thesoftware program can generate the stereographic appearance of theseimages automatically. Alternatively, the user can direct the softwareprogram to perform different modifications such that each image has auser selected stereographic appearance. Thus the software enables theuser to create his or her own stereographic display of a series ofimages which can be modified as the user desires.

9. Stereographic Display of Computer Images

In embodiments where the present invention is used to generate threedimensional stereo graphics of a computer image, the modified stereoimages may be formed by taking a source image and forming two moremodified images according to the present invention. This may be done,for example, by mapping the source image onto two wire frame meshes andthen distorting at least one of the images by the distorting one or bothof the wire frame meshes onto which the source image is mapped. Anequivalent effect to distorting the wire frame meshes can be done viamathematical manipulation of data encoding the source image.

FIG. 10 illustrates examples of different pairs of wire meshes which canbe generated by modifying the wire mesh illustrated in the figure as theinitial wire mesh. By mapping a source image onto two initial wiremeshes for the left and right eyes and then distorting one or both ofthe wire meshes as illustrated in the figure, a stereo pair of imagescan be created.

10. Image Data Management Protocol for Image Displays

In one embodiment, image data for a given line of an image is receivedand modified according to the present invention into data for a stereopair of the given line. According to this protocol, each line of thestereo pair of lines is then displayed to the left and right eyerespectively before new image data for same given line is received anddisplayed. This is accomplished by increasing the frequency at which agiven line is presented to the left or right eye. As a result, the samenumber of lines are presented over time to each of the left and righteyes as would be presented to the left and right eyes if the line datawere not modified according to the present invention. The above protocolmay be used with interlaced or noninterlaced displays known as multiplefrequency monitors.

While the present invention is disclosed by reference to the preferredembodiments and examples detailed above, it is to be understood thatthese examples are intended in an illustrative rather than limitingsense, as it is contemplated that modifications will readily occur tothose skilled in the art, which modifications will be within the spiritof the invention and the scope of the appended claims.

1. A method for producing a synthesized stereoscopic image of a sourceimage comprising: displaying at least two images which when viewed incombination form the synthesized stereoscopic image of the source image,do not simulate how a physical three dimensional model of the sourceimage would independently appear to a left eye and a right eye of anobserver, and include one image which differs from another of the atleast two images such that at least a portion of the one image ismagnified, reduced, rotated, displaced in a vertical direction, ormodified such that a position of one or more elements of that imagerelative to other elements of that image is different than relativepositionings of the corresponding elements in another of the at leasttwo images.
 2. The method according to claim 1, wherein the step ofdisplaying the at least two images includes transmitting the imagesignals encoding the at least two images to an image display.
 3. Themethod according to claim 2 wherein at least one of the image signals isan analog signal.
 4. The method according to claim 2 wherein at leastone of the image signals is a digital signal.
 5. The method according toclaim 2, wherein the image signals are transmitted to the image displayby a device which converts recorded data encoding the at least twoimages into the image signals.
 6. The method according to claim 2,wherein the image signals are transmitted to the image display by adevice which converts a signal encoding the source image into the imagesignals.
 7. The method according to claim 6, wherein converting thesignal encoding the source image into the image signals is performed bythe device in real time.
 8. The method according to claim 1, whereinonly one image of the at least two images is modified relative to thesource image.
 9. The method according to claim 1, wherein at least twoof the at least two images are modified relative to the source image.10. The method according to claim 1, wherein one of the at least twoimages is magnified or reduced relative to another of the at least twoimages.
 11. The method according to claim 1, wherein one of the at leasttwo images is rotated in the X and/or Y and/or Z plane relative toanother of the at least two images.
 12. The method according to claim 1,wherein images for a given eye are displayed at a frequency that isgreater than the frequency at which that eye can perceive individualimages.
 13. The method according to claim 1, wherein a position of oneor more elements of one of the at least two images relative to otherelements of that image is different than relative positionings of thecorresponding elements in another of the at least two images.
 14. Themethod according to claim 1, wherein at least one of the at least twoimages is transformed relative to the source image using a functionwhich alters the position of elements of the image along the Y axis. 15.The method according to claim 1, wherein wherein at least one of the atleast two images is transformed relative to the source image using afunction which is a distorting algorithm.
 16. The method according toclaim 1, wherein at least one of the at least two images is transformedrelative to the source image using a function which is an elliptical oraspheric algorithm.
 17. The method according to claim 1, wherein atleast one of the at least two images is transformed relative to thesource image using a function which is nonlinear along at least one ofthe X and Y axes.
 18. A method for viewing a synthesized stereoscopicimage comprising: displaying a source image on an image display; andviewing the source image through stereo viewing glasses, the stereoviewing glasses having left and right lenses, at least one of the lensesmodifying the source image to produce at least two images which whenviewed in combination form the synthesized stereoscopic image of thesource image, do not simulate how a physical three dimensional model ofthe source image would independently appear to a left eye and a righteye of an observer, and include one image which differs from another ofthe at least two images such that at least a portion of the one image ismagnified, reduced, rotated displaced in a vertical direction, ormodified such that a position of one or more elements of that imagerelative to other elements of that image is different than relativepositionings of the corresponding elements in another of the at leasttwo images.
 19. The method according to claim 18, wherein modifying thesource image to produce the at least two images is performed in realtime.
 20. The method according to claim 18, wherein only one of the atleast two images is modified relative to the source image.
 21. Themethod according to claim 18, wherein at least two of the at least twoimages are modified relative to the source image.
 22. The methodaccording to claim 18, wherein one of the at least two images ismagnified or reduced relative to another of the at least two images. 23.The method according to claim 18, wherein one of the at least two imagesis rotated in the X and/or Y and/or Z plane relative to another of theat least two images.
 24. The method according to claim 18, whereinimages for a given eye are displayed at a frequency that is greater thanthe frequency at which that eye can perceive individual images.
 25. Themethod according to claim 18, wherein a position of one or more elementsof one of the at least two images relative to other elements of thatimage is different than relative positionings of the correspondingelements in another of the at least two images.
 26. The method accordingto claim 18, wherein at least one of the at least two images istransformed relative to the source image using a function which altersthe position of elements of the image along the Y axis.
 27. The methodaccording to claim 18, wherein at least one of the at least two imagesis transformed relative to the source image using a function which isnonlinear along at least one of the X and Y axes.
 28. A method forsynthesizing stereoscopic image from a source image comprising: taking asignal encoding a source image and forming at least two image signalsencoding the source image; modifying at least one of the image signalssuch that the image signals encode at least two images which when viewedin combination form the synthesized stereoscopic image of the sourceimage, do not simulate how a physical three dimensional model of thesource image would independently appear to a left eye and a right eye ofan observer, and include one image which differs from another of the atleast two images such that at least a portion of the one image ismagnified, reduced, rotated, displaced in a vertical direction, ormodified such that a position of one or more elements of that imagerelative to other elements of that image is different than relativepositionings of the corresponding elements in another of the at leasttwo images; and displaying on an image display the at least two imagesencoded by the modified image signals.
 29. The method according to claim28, wherein modifying at least one of the image signals is performed inreal time.
 30. The method according to claim 28 wherein the modificationof the source image involves a signal conversion selected from the groupconsisting of digital to digital, digital to analog, analog to digitaland analog to analog.
 31. The method according to claim 28, wherein onlyone of the at least two images is modified relative to the source image.32. The method according to claim 28, wherein at least two of the atleast two images are modified relative to the source image.
 33. Themethod according to claim 28, wherein one of the at least two images ismagnified or reduced relative to another of the at least two images. 34.The method according to claim 28, wherein one of the at least two imagesis rotated in the X and/or Y and/or Z plane relative to another of theat least two images.
 35. The method according to claim 28, wherein aposition of one or more elements of one of the at least two imagesrelative to other elements of that image is different than relativepositionings of the corresponding elements in another of the at leasttwo images.
 36. The method according to claim 28, wherein at least oneof the at least two images is transformed relative to the source imageusing a function which alters the position of elements of thetransformed image along the Y axis.
 37. The method according to claim28, wherein at least one of the it least two images is transformedrelative to the source image using a function which is nonlinear alongat least one of the X and Y axes.
 38. The method according to claim 28,wherein wherein at least one of the at least two images is transformedrelative to the source image using a function which is a distortingalgorithm.
 39. The method according to claim 28, wherein at least one ofthe at least two images is transformed relative to the source imageusing a function which is an elliptical or aspheric algorithm.
 40. Themethod according to claim 28 wherein at least one of the at least twoimages includes a horizontal or vertical shift relative to the sourceimage.
 41. The method according to claim 1, wherein at least one of theat least two images is transformed relative to the source image using afunction which alters the position of elements of the image along the Xaxis.
 42. The method according to claim 1, wherein some or all of atleast one of the at least two images is transformed relative to thesource image based on speed of motion.
 43. The method according to claim1, further comprising the step of determining the position of each ofthe one or more elements of that image relative to other elements ofthat image based on variable delay.
 44. The method according to claim43, wherein determining the position of each of the one or more elementsof the transformed image relative to other elements of that image basedon variable delay comprises: determining the position of any one of theone or more elements of the transformed image based on the speed ofmotion of that one element.
 45. The method according to claim 43,wherein determining the position of each of the one or more elements ofthe transformed image relative to other elements of that image based onvariable delay comprises: determining the position of any one of the oneor more elements of the transformed image based on a functions of therelative speed of one or more of the objects and or background of theimage.
 46. The method according to claim 1, wherein the position of theone or more elements of that image are determined based on encodinginformation.
 47. The method according to claim 46, wherein the encodinginformation comprises information obtained while encoding the imagesignals.
 48. The method according to claim 18, wherein at least one ofthe at least two images is transformed relative to the source imageusing a function which alters the position of elements of the imagealong the X axis.
 49. The method according to claim 18, wherein some orall of at least one of the at least two images is transformed relativeto the source image based on speed of motion.
 50. The method accordingto claim 18, further comprising the step of determining the position ofeach of the one or more elements of that image relative to otherelements of that image based on variable delay.
 51. The method accordingto claim 50, wherein determining the position of each of the one or moreelements of that image relative to other elements of that image based onvariable delay comprises: determining the position of any one of the oneor more elements of that image based on the speed of motion of that oneelement.
 52. The method according to claim 50, wherein determining theposition of each of the one or more elements of the transformed imagerelative to other elements of that image based on variable delaycomprises: determining the position of any one of the one or moreelements of the transformed image based on a functions of the relativespeed of one or more of the objects and or background of the image. 53.The method according to claim 18, wherein the position of the one ormore elements of that image are determined based on encodinginformation.
 54. The method according to claim 53, wherein the encodinginformation comprises information obtained while encoding the imagesignals.
 55. The method according to claim 28, wherein at least one ofthe at least two images is transformed relative to the source imageusing a function which alters the position of elements of the imagealong the X axis.
 56. The method according to claim 28, wherein some orall of at least one of the at least two images is transformed relativeto the source image based on speed of motion.
 57. The method accordingto claim 28, further comprising the step of determining the position ofeach of the one or more elements of that image relative to otherelements of that image based on variable delay.
 58. The method accordingto claim 55, wherein determining the position of each of the one or moreelements of that image relative to other elements of that image based onvariable delay comprises: determining the position of any one of the oneor more elements of that image based on the speed of motion of that oneelement.
 59. The method according to claim 57, wherein determining theposition of each of the one or more elements of the transformed imagerelative to other elements of that image based on variable delaycomprises: determining the position of any one of the one or moreelements of the transformed image based on a functions of the relativespeed of one or more of the objects and or background of the image. 60.The method according to claim 28, wherein the position of the one ormore elements of that image are determined based on encodinginformation.
 61. The method according to claim 60, wherein the encodinginformation comprises information obtained while encoding the imagesignals.